• Add NemoGym Wordle GRPO training config (grpo_wordle_nemotron_nano_v2_9b.yaml)
• Add Nemotron JSON tool call parser (nemotron_json_tool_parser.py)
• Fix _replace_prefix_tokens and _postprocess_nemo_gym_to_nemo_rl_result crashes when chat templates strip reasoning tokens from prior assistant messages

Dependencies

• Requires Gym PR: Add word-guessing game environment Gym#666 (Wordle resource server)

Bug Fix: Token alignment with reasoning-stripping chat templates

Models like Nemotron Nano 9B v2 have chat templates that strip ... from prior assistant messages when re-rendering for subsequent turns. This causes two assertion failures during NemoGym multi-turn
tool-calling training:

1. _replace_prefix_tokens (vllm_worker_async.py) — template drops the last assistant message when content is empty after stripping, causing len(template_token_ids) <= len(template_prefix_token_ids) or missing EOS
2. _postprocess_nemo_gym_to_nemo_rl_result (nemo_gym.py) — token contiguity check fails because generation_token_ids includes thinking tokens but re-tokenized prompts don't

Fix: Fall back to template token IDs when alignment fails instead of crashing. Note: this causes token duplication in affected samples, which may slightly impact training quality. The proper fix would be to strip
thinking tokens from generation_token_ids before recording, matching what the template does on re-render.

Training Setup

Generate Wordle data (in Gym repo)

  cd 3rdparty/Gym-workspace/Gym/resources_servers/wordle
  python generate_data.py --output_dir data/

Run training

  uv run python examples/nemo_gym/run_grpo_nemo_gym.py \
      --config examples/nemo_gym/grpo_wordle_nemotron_nano_v2_9b.yaml